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DETAILED ACTION 

Drawings 

1. New corrected drawings in compliance with 37 CFR 1.121(d) are required 
in this application because figures 1-1 Ob, 18, and 19 are of insufficient quality for 
accurate reproduction (see MPEP 37 CFR 1 .84(a,b, and l-p). Applicant is 
advised to employ the services of a competent patent draftsperson outside the 
Office, as the U.S. Patent and Trademark Office no longer prepares new 
drawings. The corrected drawings are required in reply to the Office action to 
avoid abandonment of the application. The requirement for corrected drawings 
will not be held in abeyance. 

2. The drawings are objected to under 37 CFR 1 .83(a) because they fail to 
show units 48 and 50 in figure 22 as described in the specification on page 36, 
lines 15-17. Any structural detail that is essential for a proper understanding of 
the disclosed invention should be shown in the drawing. MPEP § 608.02(d). 
Corrected drawing sheets in compliance with 37 CFR 1 .121(d) are required in 
reply to the Office action to avoid abandonment of the application. Any amended 
replacement drawing sheet should include all of the figures appearing on the 
immediate prior version of the sheet, even if only one figure is being amended. 
The figure or figure number of an amended drawing should not be labeled as 
"amended." If a drawing figure is to be canceled, the appropriate figure must be 
removed from the replacement sheet, and where necessary, the remaining 
figures must be renumbered and appropriate changes made to the brief 
description of the several views of the drawings for consistency. Additional 
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replacement sheets may be necessary to show the renumbering of the remaining 
figures. Each drawing sheet submitted after the filing date of an application must 
be labeled in the top margin as either "Replacement Sheet" or "New Sheet" 
pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, 
the applicant will be notified and informed of any required corrective action in the 
next Office action. The objection to the drawings will not be held in abeyance. 
3. The drawings are objected to as failing to comply with 37 CFR 1 .84(p)(5) 
because they include the following reference character(s) not mentioned in the 
description: 128. Corrected drawing sheets in compliance with 37 CFR 1 .121(d), 
or amendment to the specification to add the reference character(s) in the 
description in compliance with 37 CFR 1.121(b) are required in reply to the Office 
action to avoid abandonment of the application. Any amended replacement 
drawing sheet should include all of the figures appearing on the immediate prior 
version of the sheet, even if only one figure is being amended. Each drawing 
sheet submitted after the filing date of an application must be labeled in the top 
margin as either "Replacement Sheet" or "New Sheet" pursuant to 37 CFR 
1 .121(d). If the changes are not accepted by the examiner, the applicant will be 
notified and informed of any required corrective action in the next Office action. 
The objection to the drawings will not be held in abeyance. 
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Claim Objections 

4. Claims 8, 1 1 , 26, and 29 are objected to because of the following 
informalities: 

Regarding claim 8, the further limitation of claim 7, in part seeks 
protection for "resuming said subsequent replay of said accompanying audio if a 
difference between said synchronization point and a current time value exceeds 
a second predefined amount...." The specification, on page 44, lines 19-21, 
discloses "If the difference does not exceed the predefined length of time, then 
the normal playback of the accompanying audio is resumed, at block 326." 
These two disclosures appear to contradict each other. 

Claims 11, 26, and 29 have similar limitations to claim 8, and are objected 
for the same reasons. 

Appropriate correction is required. 

5. Hereinafter, claims 8, 11, 26, and 29 are interpreted in light of the 
specification to limit the invention to "... a difference between said 
synchronization point and a current time value" does not exceed "a second 
predefined amount...." 

Claim Rejections - 35 USC §112 

6. The following is a quotation of the second paragraph of 35 U.S.C. 1 12: 

The specification shall conclude with one or more claims particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 
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7. Claims 9, 12, 27, and 30 are rejected under 35 U.S.C. 112, second 
paragraph, as being indefinite for failing to particularly point out and distinctly 
claim the subject matter which applicant regards as the invention. 

Regarding claim 9, which ultimately depends on claim 1, through claims 7 
and 8, it is unclear how "said second predefined amount equals said predefined 
amount", wherein the predefined amount is first disclosed in claim 7. 

Claim 7, the further limitation of claim 1 , in part seeks protection for: 

"comparing said synchronization with a time value associated with 

said another processing of said recorded user inputs; and 

selectively pausing said subsequent replay... if a difference 

between said synchronization point and said time exceeds a predefined 

amount...." 

Claim 8, the further limitation of claim 7, in part seeks protection for: 

"resuming said subsequent replay of said accompanying audio if a 
difference between said synchronization point and a current time value 
exceeds a second predefined amount, said current time value being 
associated with said another processing of said recorded user inputs." 
It appears that the "current time value" from claim 8 and "said time" from claim 7 
are both "associated with said another processing of said recorded user inputs", 
and therefore are the same moments in time. Therefore, it is unclear how the 
audio can be selectively paused and resumed when a value exceeds a threshold 
by the same amount. 
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Claims 12, 27, and 30 have similar limitations to claim 8, and are objected 
for the same reasons. 



Claim Rejections - 35 USC § 102 

8. The following is a quotation of the appropriate paragraphs of 35 
U.S.C. 102 that form the basis for the rejections under this section made in this 
Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in 
public use or on sale in this country, more than one year prior to the date of application for patent in 
the United States. 

(e) the invention was described in (1 ) an application for patent, published under section 
122(b), by another filed in the United States before the invention by the applicant for patent or 
(2) a patent granted on an application for patent by another filed in the United States before 
the invention by the applicant for patent, except that an international application filed under 
the treaty defined in section 351(a) shall have the effects for purposes of this subsection of an 
application filed in the United States only if the international application designated the United 
States and was published under Article 21(2) of such treaty in the English language. 

9. Claims 1-4 and 19-22 are rejected under 35 U.S.C. 102(e) as being 
clearly anticipated by Weng, US 2002/0152466 A1 . 

10. Regarding claim 1, Weng teaches a method synchronizing operations in a 
computer environment with accompanying audio, said method comprising: 

replaying said operations and said accompanying audio in said computer environment, 
said operations resulting from processing of recorded user inputs; 0004, 0051 -0055 and 
figure 4 teaches replaying operations with accompanying audio, which were 
recorded earlier (see If 0038-0041 and 0050)) 

creating a synchronization point at a common point in said replaying of said operations 
and said accompanying audio; flj 0042 and 0056 teaches a common point t2) and 

associating said synchronization point with said accompanying audio, said 
synchronization point providing a reference point to substantially synchronize said accompanying 
audio when said operations are replayed in a replay computer environment using said recorded 
user inputs. fl[ 0012 and 0068-0072 and figure 5 teaches a more generalized 
version of synchronization points and accompanying audio as taught earlier (see 
H 0041-0049 and 0055-0063)). 
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1 1 . Regarding claim 2, Weng teaches the method of claim 1 , wherein 

creating of said synchronization point includes creating said synchronization point in 
response to a user command. fl[ 0040-0042 teaches a user recording audio to be 
synchronized with a user initiated web event and an "onclick" event, which are 
user commands). 

12. Regarding claim 3, Weng teaches the method of claim 1 , wherein 

said common point is at a point in time where there is no audio output during said 
replaying of said accompanying audio. fl[ 0040 teaches a user creating the first 
synchronization point, wherein the audio has not been recorded yet). 

1 3. Regarding claim 4, Weng teaches the method of claim 1 , further 
comprising 

obtaining a current time value associated with said processing of said recorded user 
inputs, said current time value corresponding to said synchronization point. (^ 0064-0067 
teaches current time values tpN (where N=1 ,2,3,or 4) corresponding to different 
synchronization points (see figure 4)). 

14. Regarding claim 19, see the preceding argument with respect to claim 1 . 
Weng teaches the method of claim 1 , which reads on the storage medium 
readable by a computer, tangibly embodying a program of instructions 
executable by said computer to perform method steps for synchronizing 
operations in a computer environment with accompanying audio, said method 
comprising the features of claim 1 . 

1 5. Regarding claim 20, see the preceding argument with respect to claim 2. 
Weng teaches the features of claim 2, which reads on the features of claim 20. 

16. Regarding claim 21 , see the preceding argument with respect to claim 3. 
Weng teaches the features of claim 3, which reads on the features of claim 21 . 

1 7. Regarding claim 22, see the preceding argument with respect to claim 4. 
Weng teaches the features of claim 4, which reads on the features of claim 22. 
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18. Claims 1, 2, 4, 7, 10, 14, 15, 19, 20, 22, 25, 28, 32, and 33 are rejected 
under 35 U.S.C. 102(b) as being clearly anticipated by Ishiguro, US 6,180,865 
B1. 

19. Regarding claim 1, Ishiguro teaches a method synchronizing operations 

in a computer environment with accompanying audio, said method comprising: 

replaying said operations and said accompanying audio in said computer environment, 
said operations resulting from processing of recorded user inputs; (column 2, lines 22-38 
and column 5, lines 34-38 teaches MIDI operations, which have been previously 
recorded, to be reproduced in part by input from a user and accompanying 
audio) 

creating a synchronization point at a common point in said replaying of said operations 
and said accompanying audio; (column 2, lines 39-54 teaches two different results 
with respect to a common point) and 

associating said synchronization point with said accompanying audio, said 
synchronization point providing a reference point to substantially synchronize said accompanying 
audio when said operations are replayed in a replay computer environment using said recorded 
user inputs, (column 2, lines 39-54 teaches accompanying audio to substantially 
synchronize with recorded user inputs). 

20. Regarding claim 2, Ishiguro teaches the method of claim 1 , wherein 

creating of said synchronization point includes creating said synchronization point in 
response to a user command, (column 2, lines 39-54 teaches a synchronization point 
being created when the key is actuated early). 

21 . Regarding claim 4, Ishiguro teaches the method of claim 1 , further 
comprising 

obtaining a current time value associated with said processing of said recorded user 
inputs, said current time value corresponding to said synchronization point, (column 1 , lines 
18-22 teaches a synchronization point corresponding to a current time value (i.e. 
the key is pressed at the correct time)). 

22. Regarding claim 7, Ishiguro teaches the method of claim 1 , further 
comprising 

detecting said synchronization point during a subsequent replay of said operations and 
said accompanying audio in said replay computer environment, said subsequent replay involving 
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another processing of said recorded user inputs; (column 2, lines 22-38 wherein the song 

can be replayed a second time) 

comparing said synchronization point with a time value associated with said another 
processing of said recorded user inputs; (column 2, lines 39-54) and 

selectively pausing said subsequent replay of said accompanying audio if a difference 
between said synchronization point and said time value exceeds a predefined amount so that 
said subsequent replay of said operations can catch up to said accompanying audio, (column 
2, lines 39-54 teaches pausing the melody if the proper key is not depressed in 
time). 

23. Regarding claim 10, see the preceding argument with respect to claim 1 
and 7. Ishiguro teaches the similar methods of claim 1 and claim 7. 

24. Regarding claim 14, the further limitation of claim 10, see the preceding 
argument with respect to claim 1 . Ishiguro teaches the features of claim 1 0 and 
these additional features. 

25. Regarding claim 15, the further limitation of claim 14, see the preceding 
argument with respect to claim 2. Ishiguro teaches the features of claim 14 and 
these additional features. 

26. Regarding claim 19, see the preceding argument with respect to claim 1 . 
Ishiguro teaches the method of claim 1 , which reads on the storage medium 
readable by a computer, tangibly embodying a program of instructions 
executable by said computer to perform method steps for synchronizing 
operations in a computer environment with accompanying audio, said method 
comprising the features of claim 1 . 

27. Regarding claim 20, the further limitation of claim 19, see the preceding 
argument with respect to claim 2. Ishiguro teaches these features. 

28. Regarding claim 22, the further limitation of claim 19, see the preceding 
argument with respect to claim 4. Ishiguro teaches these features. 
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29. Regarding claim 25, the further limitation of claim 19, see the preceding 
argument with respect to claim 7. Ishiguro teaches these features. 

30. Regarding claim 28, see the preceding argument with respect to claims 1 
and 7. Ishiguro teaches the methods of claim 1 and 7, which reads on the 
storage medium readable by a computer, tangibly embodying a program of 
instructions executable by said computer to perform method steps for 
synchronizing operations in a computer environment with accompanying audio, 
said method comprising the combined features of claim 1 and 7. 

31 . Regarding claim 32, the further limitation of claim 28, see the preceding 
argument with respect to claim 1 . Ishiguro teaches these features. 

32. Regarding claim 33, the further limitation of claim 32, see the preceding 
argument with respect to claim 2. Ishiguro teaches these features. 

Claim Rejections - 35 USC § 103 

33. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for 
all obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described 
as set forth in section 102 of this title, if the differences between the subject matter sought to 
be patented and the prior art are such that the subject matter as a whole would have been 
obvious at the time the invention was made to a person having ordinary skill in the art to which 
said subject matter pertains. Patentability shall not be negatived by the manner in which the 
invention was made. 

34. Claims 5 and 23 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Weng as applied to claim 1 above, and further in view of well- 
known prior art. 
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35. Regarding claim 5, Weng teaches the method of claim 1 . Specifically, 
Weng teaches various different containers (i.e. software programs) for loading 
article content 0007-0012), wherein operation are replayed with accompanying 
audio. Weng does not specifically teach one file for the accompanying audio and 
a second separate file for the recorded user inputs, however the Office takes 
Official Notice that it is well-known at the time of the invention by one of ordinary 
skill in the art to separate different types of data into different files. Webpages 
have been known to embed different types of data, such as images and audio, 
wherein the text of the webpage is contained in one file (i.e. a hyperlink mark-up 
text language (HTML) file) and an image or sound is contained in another file 
(e.g. a JPEG file or an MP3 file, respectively). It would have been obvious for 
one of ordinary skill in the art at the time of the invention to combine the 
teachings of Weng and the well-known prior art for the purpose of following 
established standards for placing different content in respective files. 

36. Regarding claim 23, the further limitation of claim 19, see the preceding 
argument with respect to claim 5. The combination teaches these features. 

37. Claims 6, 13, 18, 24, 31 and 36 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over Ishiguro as applied to claim 1 above, and further in view 
of Sung et al., US 6,423,893 B1 (hereinafter Sung). 

38. Regarding claim 6, the further limitation of claim 1 , see the preceding 
argument with respect to claim 1 . Ishiguro teaches the features of claim 1 , but 



Application/Control Number: 10/770,935 Page 12 

Art Unit: 2615 

does not teach changing a time value of said synchronization point in response 
to a positional change of a marker of said synchronization point in a timeline. 

Sung teaches editing MIDI data for a solo instrument part on a timeline 
(column 7, lines 46-61 and figure 7a, unit 74). It is obvious to extend this idea to 
any of the instruments in a MIDI composition and edit the notes on a timeline, 
which has the same effect as changing a marker's position. It would have been 
obvious for one of ordinary skill in the art at the time of the invention to combine 
the teachings of Ishiguro and Sung for the purpose of editing MIDI data 
graphically for ease of use. 

39. Regarding claim 13, the further limitation of claim 10, see the preceding 
argument with respect to claim 10. Ishiguro teaches the features of claim 10, but 
does not teach displaying the synchronization point as a marker on a timeline, 
where the timeline includes time values extracted from the recorded user inputs. 
Sung teaches a display of synchronization points, or notes, as markers (see 
figure 7a), wherein time values are extracted from recorded user inputs (i.e. the 
mouse is used to create the markers with various information and MIDI data is 
created from the meta-data, see column 7, line 46 - column 8, line 10). 

40. Regarding claim 18, the further limitation of claim 14, see the preceding 
argument with respect to claim 6. The combination teaches these features. 

41 . Regarding claim 24, the further limitation of claim 19, see the preceding 
argument with respect to claim 6. The combination teaches these features. 

42. Regarding claim 31 , the further limitation of claim 28, see the preceding 
argument with respect to claim 13. The combination teaches these features. 
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43. Regarding claim 36, the further limitation of claim 32, see the preceding 
argument with respect to claim 6. The combination teaches these features. 

Allowable Subject Matter 

44. Claims 8, 9, 11, 12, 16, 17, 26, 27, 29, 30, 34, and 35 are objected to as 
being dependent upon a rejected base claim, but would be allowable if rewritten 
in independent form including all of the limitations of the base claim and any 
intervening claims. 

Conclusion 

45. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

Owen, US 5,585,583 A, teaches mixing a student's musical instrument 
with a synchronized audio/video playback of an instructional musical piece 
(abstract, column 3, line 57 - column 4, line 9, column 6, lines 43-59, column 7, 
lines 15-26, column 7, line 56 - column 8, line 16 and column 13, lines 48); 

Kuribayashi, US 5,615,189 A, teaches synchronization of a word clock in 
MIDI circuits (abstract and column 1 , line 50 - column 2, line 20); 

Berstis et al., US 5,903,266 A, teaches audio instructions detailing various 
steps of connecting equipment (column 5, line 48 - column 8, line 22 and figures 
3 and 4); 



Application/Control Number: 10/770,935 Page 
Art Unit: 2615 

Falcon et al., US 6,632,094 B1, teaches synchronized text highlighting 
with audio narration for improving reading skills (abstract and column 8, lines 23- 
26); 

Sastry et al., US 6,687,877 B1 , teaches content annotation synchronized 
with user actions and recorded audio (column 5, line s 39-61); 

Blume, US 6,915,103 B2, teaches an audio broadcast system to read an 
audio track corresponding to the position and speed of a handheld stylus (column 
3, line 32-56 and figures 1-5); 

Lilienthal, US 6,933,928 B1 , teaches a paperless book with visual text that 
advances as the audio text is finished (column 3, lines 43-50); and 

Gaddy et al., US 7,228,189 B1, teaches a method for synchronizing a 
streaming music file with a locally captured audio input (abstract, column 7, line 
39 - column 8, line 36, and figures 3a and 3b). 

Any inquiry concerning this communication or earlier communications from 
the examiner should be directed to DANIEL R. SELLERS whose telephone 
number is (571)272-7528. The examiner can normally be reached on Monday to 
Friday, 9am to 5:30pm. 

If attempts to reach the examiner by telephone are unsuccessful, the 
examiner's supervisor, Sinh Tran can be reached on (571)272-7564. The fax 
phone number for the organization where this application or proceeding is 
assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from 
the Patent Application Information Retrieval (PAIR) system. Status information 
for published applications may be obtained from either Private PAIR or Public 
PAIR. Status information for unpublished applications is available through 
Private PAIR only. For more information about the PAIR system, see http://pair- 
direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll- 
free). If you would like assistance from a USPTO Customer Service 
Representative or access to the automated information system, call 800-786- 
9199 (IN USA OR CANADA) or 571-272-1000. 



/Daniel R. Sellers/ 
Examiner, Art Unit 2615 

/Sinh N Tran/ 

Supervisory Patent Examiner, Art Unit 2615 



